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Univariate  Approximation 

••  f(x),  x  G  [0, 1] 

»  Approximate  /  by  piecewise  constants 
3  Types  of  approximation:  Linear,  Nonlinear 
3  Select  a  metric/norm  II  •  II  to  measure  distortion 


0  <  p  <  co 
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Linear  approximation 

»  Divide  [0, 1]  into  n  equal  length  intervals 
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Linear  approximation 

*  Divide  [0, 1]  into  n  equal  length  intervals 

*  n„:={[£,*±i),  k  =  0, . . .  ,n  —  1} 

.•  Sn  :=  {S  :  S  is  constant  on  each  I  G  IIn} 

*  Sn  is  a  linear  space 
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Typical  function  in  Sn 
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1  1 

N  N 


N-1 
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Linear  approximation 

*  Divide  [0, 1]  into  n  equal  length  intervals 

•  n„:={[£,*±i),  k  =  0, . . .  ,n  —  1} 

3  Sn  :=  {S  :  S  is  constant  on  each  I  G  IIn} 

3  Sn  is  a  linear  space 
3  Given  /  e  Lp[ 0,  l],  define  error 

En(f)P  ■=  jnf  ||/  -  S\\Lp 

Gun 
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Nonlinear:  Free  Knot  Splines 


Divide  [0, 1]  into  n  intervals  k  =  [&,&+i], 

0  =  Co  <  £l  <  •  •  •  <  £n  =  1 


Nonlinear:  Free  Knot  Splines 


Divide  [0, 1]  into  n  intervals  k  =  [&,&+i], 

0  =  Co  <  £l  <  •  •  •  <  £n  =  1 

En  :=  {S  :  S  is  constant  on  each  Ii,  i  =  1, . . . ,  n} 
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Nonlinear:  Free  Knot  Splines 
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Nonlinear:  Free  Knot  Splines 


.*  Divide  [0, 1]  into  n  intervals  k  =  [&,&+i], 

0  =  Co  <  Cl  <  •  •  •  <  =  1 

.•  Y]n  :=  {S  :  S  is  constant  on  each  U,i  =  1, . . . ,  n} 

*  £n  is  not  a  linear  space:  Si,  S2  e  En  but  Si  +  S2  e  T,2n 

.#  Given  /  e  Lp[ 0, 1],  define  error 


II/-SIU, 
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Nonlinear:  Adaptive 


3  For  each  interval  I,  E(I)  the  local  Lp  error  in 
approximating  /  by  constants 


E(I)  :=  inf  ||/ 

C 
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LP(I ) 


Cargese  -  p.7/4S 


Nonlinear:  Adaptive 


For  each  interval  I,  E(I)  the  local  Lp  error  in 
approximating  /  by  constants 


E(I)  :=  inf  ||/ 

C 


C 


LP(I ) 


Given  error  tolerance  e  >  0  generate  partition  Ve  such 
that  E(I)  <  e  for  all  I  eVe 


Cargese  -  p.7/4S 


Nonlinear:  Adaptive 


For  each  interval  I,  E(I)  the  local  Lp  error  in 
approximating  /  by  constants 


E(I)  :=  inf  ||/ 

C 


C 


LP(I ) 


Given  error  tolerance  e  >  0  generate  partition  Ve  such 
that  E(I)  <  e  for  all  I  eVt 

I  is  good  if  E(I)  <  e 


Cargese  -  p.7/4£ 


Nonlinear:  Adaptive 


For  each  interval  I,  E(I)  the  local  Lp  error  in 
approximating  /  by  constants 


E(I)  :=  inf  ||/ 

C 


C 


LP(I ) 


Given  error  tolerance  e  >  0  generate  partition  Ve  such 
that  E(I)  <  e  for  all  I  eVt 

I  is  good  if  E(I)  <  e 

I  is  bad  if  E{l)  >  e 
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Nonlinear:  Adaptive 


Initially  if  E([0, 1])  <  e  then  algorithm  terminates  and 

^:={[0,l]} 
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Nonlinear:  Adaptive 


Initially  if  E([0, 1])  <  e  then  algorithm  terminates  and 

^:={[0,l]} 

if  E([ 0, 1])  >  e  then  set  Be  •—  {[0, 1]}  and  Qt  ■-  0 

Recursion:  For  each  I  e  Be  put  child  J  of  /  in  if  it  is 
good,  put  it  in  Be  if  it  is  bad.  Remove  /  from  Be 
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Nonlinear:  Adaptive 


Initially  if  E([0, 1])  <  e  then  algorithm  terminates  and 

^:={[0,l]} 

if  E([ 0, 1])  >  e  then  set  Be  •—  {[0, 1]}  and  Qt  ■-  0 

Recursion:  For  each  I  e  Be  put  child  J  of  /  in  if  it  is 
good,  put  it  in  Be  if  it  is  bad.  Remove  /  from  Be 

Stop  when  Be  =  0,  Ve  :=  Qe,  Ne  :=  #(V£) 
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Nonlinear:  Adaptive 


ANe(f )  best  approximation  to  /  by  piecewise  constants 
on  Ve 
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Nonlinear:  Adaptive 


ANe(f )  best  approximation  to  /  by  piecewise  constants 
on  Ve 

M/)p  ■=  inf {e  :  Ne<n} 
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Adaptively  generated  partition 
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1  3 

4  8 
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Tree  associated  to  adaptive  partition 


[1/2,1] 
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Comparison 


Approximation  classes:  a  >  0 

define^ta(Lp,  linear  splines )  as  the  set  of  all  f  e  Lp[ o,  l] 

such  that 

En{f)P  <  Cn~a,  n  >  1 
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Comparison 


Approximation  classes:  a  >  0 

define*4a(Lp,  linear  splines )  as  the  set  of  all  f  e  Lp[ o,  l] 

such  that 

En{f)P  <  Cn~a,  n  >  1 


Similarly  define  Aa(Lp)  for  the  other  forms  of 
approximation 

Aq(Lp)  finer  scaling:  same  approximation  order  a 

oo 

i/u?(w  := 

n= 1 
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Approximation  Classes:  Linear 

Fix  the  Lp  space  to  measure  error 
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Approximation  Classes:  Linear 


Fix  the  Lp  space  to  measure  error 

As(Lp,  linear )  =  B^Lp) 
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Approximation  Classes:  Linear 


Fix  the  Lp  space  to  measure  error 

As(Lp,  linear )  =  B^^Lp) 

Proved  by  Scherer  + 
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jproximation:  A^Lp)  Besov  space  of  sm< 


a 

. . . . . j 

L 

(l/p,0)(Lp) 
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Approximation  Classes:  free  knot  splines 


.#  Fix  the  Lp  space  to  measure  error 
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Approximation  Classes:  free  knot  splines 


.#  Fix  the  Lp  space  to  measure  error 

.•  A^(Lp,  nonlinear )  =  B^{LT),  ^  =  s  +  ^ 

»  Petrushev,  DeVore-Popov  (splines); 
DeVore-Jawerth-Popov  (wavelets) 
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Approximation  class:  free  knot  splines 
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Adaptive  approximation 


Cargese 


Example:  Approximation  in  L ^ 

p  =  oo  approximation  order  0{n~l) 
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Example:  Approximation  in  L ^ 


p  =  oo  approximation  order  0{n~l) 

Linear  approximation  f  e  Loo 

Nonlinear  approximation  (free  knot  splines)  /'  e  L\  or 

feBV 

Adaptive  approximation  f  e  LlogL :  for  example  f  e  Lp 
for  some  p  >  1 
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Example:  f(x)  =  xa ,  0  <  a  <  i  -  i/p 


En(f)p  ~  Cn  (a+1'  (Tn(f)p  <  Cn  1 

Break  points/  wavelets  concentrate  near  singularity  at  0 
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Example:  piecewise  smooth 


II  II 

0  x2 

1  1  1 

*N-1  1 

En(f)P  >  Cn  1/1  <7n(f)p  <  Cn  1 

Breakpoints/wavelets  concentrate  near  singularities 
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Wavelets:  Haar  Wavelet 


H(x)  := 
’r=] 


-1,  x  6  [0,1/2) 

+l,  x  e  [1/2,  l]  , 


.1 

i 

+1 

1 

0 

1 

1 1 

i  -l 
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Wavelets: 


»  Ht(x)  :=  2^2H(2^x  -  k ) 


Haar  Basis 

/=  [k2~i ,  {k  +  l)2~i] 
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Wavelets: 


H^x)  :=  2i/2H{2iX  -  k) 
V+  :=  {/  G  V  :  |/|  <  1} 


Haar  Basis 

I  =  [k2~i ,  {k  +  l)2~i] 
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Wavelets:  Haar  Basis 


H^x)  :=  2i/2H(2ix  -  k),  I  =  [k2~i ,  (k  +  l)2~i] 

V+  :=  {/  e  V  :  |/|  <  1} 

{x[o,i]}  U  {#/}/££.+  is  a  complete  orthonormal  system  in 

Hi  1] 
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Haar  Basis 
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Wavelet  tree 
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Linear  Approximation  with  Haar  Basis 


.#  Natural  ordering  of  dyadic  intervals 
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Linear  Approximation  with  Haar  Basis 


.#  Natural  ordering  of  dyadic  intervals 

*  Xn  span  of  first  n  Haar  functions 

*  Xn  is  a  linear  space 

»  E%(f)p  :=  mf9£jc„  ||/  -  Silvio.!, 


Cargese  -  p.25/42 


Linear  Approximation  with  Haar  Basis 


.#  Natural  ordering  of  dyadic  intervals 

*  Xn  span  of  first  n  Haar  functions 

*  Xn  is  a  linear  space 

*  E£(f)p:=  migeXn\\f  -  g\\Lp[o,i] 

3  This  is  linear  approximation  because  Xn  is  a  linear 
space 
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Linear  Approximation  with  Haar  Basis 


.#  Natural  ordering  of  dyadic  intervals 

*  Xn  span  of  first  n  Haar  functions 

*  Xn  is  a  linear  space 

*  E£(f)p:=  migeXn\\f  -  g\\Lp[o,i] 

3  This  is  linear  approximation  because  Xn  is  a  linear 
space 

3  The  approximation  classes  for  linear  approximation  with 
Haar  wavelets  are  identical  to  those  with  piecewise 
constants. 
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Linear  Wavelet:  A^Lp)  =  ( Lv ) 


a 

. . . . . j 

L 

(l/p,0)(Lp) 
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n-term  approximation 
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Nonlinear  Approximation  with  Haar  Basi: 


.#  n-term  approximation 
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Nonlinear  Approximation  with  Haar  Basi: 


.#  n-term  approximation 

3  Sn  :=  {S  =  E/6  A  ciHj  :  #(A)  <  n} 
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Nonlinear  Approximation  with  Haar  Basi: 


.#  n-term  approximation 
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*  En  is  not  a  linear  space 
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Nonlinear  Approximation  with  Haar  Basi: 
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Nonlinear  Approximation  with  Haar  Basi: 


.#  n-term  approximation 
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*  En  is  not  a  linear  space 
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.»  This  is  nonlinear  approximation  because  decisions  are 
made  dependent  on  / 
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Nonlinear  Approximation  with  Haar  Basi: 


.#  n-term  approximation 

»  Sn  :=  {S  =  E/6A  ciHi  :  #(A)  <  n} 

*  En  is  not  a  linear  space 

*  <(/)p  :=  infseEn  11/  -  S||lp[o,i] 

.»  This  is  nonlinear  approximation  because  decisions  are 
made  dependent  on  / 

»  The  approximation  classes  for  nonlinear  approximation 
with  Haar  wavelets  are  identical  to  those  with  piecewise 
constants. 
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iroximation  class  n-term  Haar  approxima 
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So  Why  Wavelets?? 


*  Simplicity  of  numerical  implementation 
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So  Why  Wavelets?? 


3  Simplicity  of  numerical  implementation 
3  How  can  we  find  best  n-term  approximation?? 
3  In  L2  take  n  terms  with  largest  coefficients 
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So  Why  Wavelets?? 


Simplicity  of  numerical  implementation 

How  can  we  find  best  n-term  approximation?? 

In  Z/2 1 ake  n  terms  with  largest  coefficients 

DJP:  Same  strategy  works  in  Lp,  1  <  p  <  oo,  and  other 
spaces  (Sobolev) 


Cargese  -  p.30/42 


Greedy  Approximation  in  X  =  Lp 

*  Write  /  =  T,ievci(f)^i with  UA\x  =  1 
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Greedy  Approximation  in  X  =  Lp 


*  Write  /  =  E762>cH/)^7  with  H^llx  =  l 

*  A n(f)  indicies  of  n  largest  coefficients  |c7(/) 
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Greedy  Approximation  in  X  =  Lp 


3  Write  /  =  Y,ievciWi with  UA\x  =  l 
3  A „(/)  indicies  of  n  largest  coefficients  |c7(/) 

*  Gn(Z)  :=  E/gA„(/) 
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Greedy  Approximation  in  X  =  Lp 
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Greedy  Approximation  in  X  =  Lp 


3  Write  /  =  Y,ievciWi with  UA\x  =  l 
3  A „(/)  indicies  of  n  largest  coefficients  |c7(/) 

*  Gn(Z)  :=  E/gA„(/) 

.#  Temlyakov: 

11/  -  Gn{f)\\x  <  Cx<?n(f)x 

3  Greedy  strategy  is  near  optimal 
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Thresholding  is  near  optimal  in  Lp 

*  A (/, 77)  :=  U  '■  \ci(f)\  >  v}>  N  :=  #(A (/, 77)) 
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Thresholding  is  near  optimal  in  Lp 


•  A(/,l>)  :=  {/  :  \c,(f)\  >  I)},  N  :=  #(A(/,i;)) 

*  T„(f) 
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Thresholding  is  near  optimal  in  Lp 

*  A (/, 77)  :=  i1  :  M/)|  >  V},  AT  :=  #(A(/, 77)) 

*  W)  :=  £/6A(/,,)  c/(/)^ 

*  11/  -  t»?(/)IIlp  <  CpaN(f)p 
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Generalizations 


i  la  Banach  space:  {^j}  basis 
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Generalizations 


i  la  Banach  space:  {^j}  basis 

*  f  =  ET=  ici(% 

.#  A „(/)  set  of  indices  of  n  largest  |cj(/) 
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3  A „(/)  set  of  indices  of  n  largest  |cj(/)| 
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.#  Konjagin-Temlyakov:  Near  optimal  equivalent  to  the 
basis  is  unconditional  and  democratic 

.#  Democratic 


IIE/ga^/ 

X 

II  E/e  A' 

X 

whenever  #(A)  =  #(A') 
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Wavelet  Bases 


.#  Wavelet  bases  are  democratic  in  Lp,  l  <  p  <  oo,  in  Hp, 
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Wavelet  Bases 


Wavelet  bases  are  democratic  in  Lp,  l  <  p  <  oo,  in  Hp, 

p  <  l 


Cl  min  |ci(/)|(#(A)1/>’  <  ||  ^  Cj(f  )4>jhr 
3€A  j€A 

\\^2ci(f)^jhr  <  c2  max  |cj  (/ )  |  (#  (A) I/,? 


je  A 


jeA 
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When  do  we  get  °n(f) 


T 


P  =  0(n  “) 
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v 


:=  (££: 


c 


r^l/r 


IKCjVIUt  VZ^j=l 

rearrangement  of  ( cj ) : 
c*  the  n-th  largest  of  | cy  | ,  j  e  {1,2,. 

weak-£T  : 

c*  <  Mn_1/r 


'n  — 


smallest  M  gives  |(c,-) 


wL, 


0(n  a ) 
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1 1 
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Approximation  properties  analogous  to  adaptive 
approximation 
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Tree  approximation 


Cargese 


Extensions 

Can  replace  Haar  wavelets  by  biorthogonal  wavelets 
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Multidimensional:  Results  hold  in  Rd  with  basis  ?/’f 

Wavelet  Tree  Approximation  equivalent  to  adaptive 
triangulation  with  isotropic  partitioning 

Aa/d(Lr)  =  (Lp) 

Nonlinear  Approximation? 

No  analogue  of  free  knot  splines:  general  triangulation 

n-term  approximation  using  multivariate  wavelets  the 
same 

Ar/d (wavelets,  Lp)  =  B?(LP),  \  =  f  ^ 
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How  can  we  evaluate  encoders 
Experimental: 

Encoders  designed  on  heuristics 

Precise  Mathematical  Formulation 

Understand  rules  of  game;  what  it  means  to  be  a  winner 

Two  essential  ingredients 

a.  metric  p  to  measure  distortion 

b.  Precise  definition  of  classes  Ka  to  be  compressed 
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Mathematical  Formulation 

.#  Distortion:  p(S,DnEn(S )) 

*  Evaluate  Performance  on  a  set  K  of  surfaces 

S(K;Dn,En)  :=  sup  p(S,  DnEn(S)) 

SeK 

m  Given  bit  budget  n 

$n{K)  inf  S(K,  DnEn(S)) 

-E'n  'Dri 

*  smallest  distortion  for  the  given  bit  budget 
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(Near)  Optimal  Encoding  for  K 

optimal 

5(K,DnEn(S))=6n(K ) 
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»  optimal 

5(K,DnEn(S))=6n(K ) 

*  near  optimal 

5(K,DnEn(S))  <  C5n(K) 
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(Near)  Optimal  Encoding  for  K 


3  optimal 


5(K,DnEn(S))  =  5n(K) 


3  near  optimal 


S(K,DnEn(S))  <  C5n(K) 

3  Typically:  5n(K)  &  n~s  for  some  s  >  0 
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(Near)  Optimal  Encoding  for  K 


.#  optimal 

5(K,DnEn(S))=Sn{K ) 

*  near  optimal 

5(K,DnEn(S))  <  C5n(K) 

*  Typically:  5n(K)  &  n~s  for  some  s  >  0 

.#  Game:  Find  encoder/decoder  E/D:  for  all  values  of  n 
and  all  classes  Ka,  encoder  is  near  optimal 
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*  Given  e  >  0 


Cargese  -  p.43/42 


Description  of  Optimal  Encoding:  Kolmo 


*  Given  e  >  0 

»  Minimal  e  cover:  K  c  u^1i5(5,*,e) 
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Description  of  Optimal  Encoding 

m  Given  e  >  0 

m  Minimal  e  cover:  K  c  u e) 

3  Kolmogorov  Entropy  He(K)  :=  log2  Ne(K) 
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Covering 
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Covering 
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Kolmogorov  Entropy 


*  Given  e  >  0 

*  Minimal  e  cover:  K  c  u^1i5(5,*,e) 

.#  Kolmogorov  Entropy  He(K)  :=  log2  Ne(K) 

*  Sn(K)  =  inf{e  :  He(K)  <  n] 
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Description  of  Optimal  Encoding:  Kolmo 


3  Given  e  >  0 
3  Minimal  e  cover:  K  c  u 

3  5n(K)  =  inf{e  :  He(K)  <  n 

3  Kolmogorov  entropy  of  K  gives  our  benchmark 
3  Usually  not  practical  encoder 
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The  Issues 


1 .  The  metric:  least  squares 

2.  The  classes 

3.  Determine  Entropy  of  Classes 

4.  Build  near  optimal  Encoders/Decoders 
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Wavelet  Encoding 

Tree  approximation  can  be  turned  into  an  encoder 
Tree  with  n  nodes  can  be  encoded  with  n  bits 

Cohen-Dahmen-Daubechies-DeVore 

(Cohen-Daubechies-Gulleryuz-Orchard)  This  encoder 
is  optimal  on  all  Besov  classes  compactly  embedded 
into  Z/2 

EZW,  Said-Pearlman, 
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